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I. INTRODUCTION 

A. General Introduction 

Training is a major endeavor in all modern societies: new 
personnel must be trained to perform the task(s) which they were 
hired to perform, continuing personnel must be trained to upgrade 
or update their ability to perform assigned tasks, and continuing 
personnel must be trained to tackle new tasks. Common methods 
include training manuals, formal classes, procedural computer 
programs, simulations, and on-the-job training. The latter method 
is particularly effective in complex tasks where a great deal of 
independence is granted to the task performer. Of course, this 
training method is also the most expensive and may be impractical 
when there are many trainees and few experienced personnel to 
conduct on-the-job training. 

NASA's training approach has focussed primarily on on-the— job 
training in a simulation environment for both crew and ground- 
based personnel. This process worked relatively well for both the 
Apollo and Space Shuttle programs. Space Station Freedom and 
other long range space exploration programs coupled with limited 
resources dictate that NASA explore new approaches to training for 
the 1990 's and beyond. 

This report describes specific autonomous training systems 
based on artificial intelligence technology for use by NASA 
astronauts, flight controllers, and ground-based support personnel 
that demonstrate an an alternative to current training systems. 
In addition to these specific systems, the evolution of a general 
architecture for autonomous intelligent training systems that 
integrates many of the features of "traditional" training programs 
with artificial intelligence techniques is presented. These 
Intelligent Computer-Aided Training (ICAT) systems would provide, 
for the trainee, much of the same experience that could be gained 
from the best on-the-job training. By integrating domain 
expertise with a knowledge of appropriate training methods, an 
ICAT session should duplicate, as closely as possible, the trainee 
undergoing on-the-job training in the task environment, benefiting 
from the full attention of a task expert who is also an expert 
trainer. Thus, the philosophy of the ICAT system is to emulate 
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the behavior of an experienced individual devoting his full time 
and attention to the training of a novice — proposing challenging 
training scenarios, monitoring and evaluating the actions of the 
trainee, providing meaningful comments in response to trainee 
errors, responding to trainee requests for information, giving 
hints (if appropriate), and remembering the strengths and 
weaknesses displayed by the trainee so that appropriate future 
exercises can be designed. 

B . BACKGROUND 

Since the 1970's a number of academic and industrial 
researchers have explored the application of artificial 
intelligence concepts to the task of teaching a variety of 
subjects [Sleeman and Brown, 1982; Yazdani, 1986; Wenger, 1987] 
(e.g., computer programming in Lisp (Anderson, 1985; Anderson, 
Boyle and Reiser, 1985] and Pascal [Johnson and Soloway, 1985], 
economics [Shute and Bonar, 1986], geography [Carbonell, 1970], 
and geometry [Anderson, Boyle and Yost, 1985]). The earliest 
published reports which suggested the applications of artificial 
intelligence concepts to teaching tasks appeared in the early 
1970's [Carbonell, 1970; Hartley and Sleeman, 1973]. Hartley and 
Sleeman [Hartley and Sleeman, 1973] actually proposed an 
architecture for an intelligent tutoring system. However, it is 
interesting to note that, in the sixteen years which have passed 
since the appearance of the Hartley and Sleeman proposal, no 
agreement has been reached among researchers on a general 
architecture for intelligent tutoring systems [Yazdani, 1986] . 

Along with the extensive work on intelligent tutoring systems 
for academic settings has come the development of systems directed 
at training. Among these are Recovery Boiler Tutor [Woolf, 
Blegen, Jansen, and Verloop, 1986], SOPHIE [Brown, Burton and de 
Kleer, 1982], and STEAMER [Hollan, Hutchins and Weitzman, 1984]. 
These differ from the tutoring systems mentioned above in 
providing a simulation model with which the student or trainee 
interacts. Although these intelligent training systems each use 
the interactive simulation approach, they each have very different 
internal architectures. Further, there appears to be no 
agreement, at present, on a general architecture for such 
simulation training systems. The work reported here builds on 
these previous efforts and our own work [Loftin, Wang, Baffes and 
Rua, 1987; Loftin, Wang, Baffes, and Hua, 1988; Loftin, Wang, 
Baffes, and Hua, 1989a and b] to develop specific intelligent 
training systems as well as a general approach to the design of 
intelligent training systems which will permit the production of 
such systems for a variety of tasks and task environments with 
significantly less effort that is now required to ''craft" such a 
system for each application. 

C. TRAINING VERSUS TUTORING 

The ICAT systems and architecture described here have been 
developed with a clear understanding that training is not the same 
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as teaching or tutoring [Harmon, 1987] . An industrial or 
governmental training environment differs in many ways from an 
academic teaching environment These differences are important in 
' the design of an architecture for an intelligent training system: 

• Assigned tasks are often mission-critical, placing the 
responsibility for lives and property in the hands of those 
who have been trained. 

• Personnel may already have significant academic and 
practical experience to bring to bear on their assigned 
task . 

• Trainees make use of a wide variety of training techniques, 
ranging from the study of comprehensive training manuals to 
simulations to actual on-the-job training under the 
supervision of more experienced personnel. 

• Many of the tasks offer considerable freedom in the exact 
manner in which they may be accomplished. 

Those undergoing training for complex tasks are usually well 
aware of the importance of their job and the probable consequences 
of failure. While students are often motivated by the fear of 
receiving a low grade, trainees know that human lives and/or 
expensive equipment may depend on their skill in performing 
assigned tasks. This means that trainees may be highly motivated, 
but it also imposes on the trainer the responsibility for the 
accuracy of the training content (i.e., verification of the domain 
expertise encoded in the system) and the ability of the trainer to 
correctly evaluate trainee actions. The ICAT approach is 
intended, not to impart basic knowledge, but to aid the trainee in 
developing skills for which he already has the basic or 
"theoretical" knowledge. In short, this training system 
architecture is designed to help a trainee put into practice that 
which he already intellectually understands. The system must take 
into account the type of training that both precedes and follows, 
building on the knowledge gained from training manuals and rule 
books while preparing the trainee for and complementing the on- 
the-job training which may follow. Perhaps most critical of all, 
trainees must be allowed to carry out an assigned task by any 
valid means. Such flexibility is essential so that trainees are 
able to retain, and even hone, an independence of thought and 
develop confidence in their ability to respond to problems, even 
problems which they have never encountered and which their 
trainers never anticipated. 

IV. APPLICATIONS 

The ICAT architecture was originally applied to a training 
system for NASA flight controllers learning to deploy satellites 
from the Space Shuttle. The same architecture has been used in 
the construction of ICAT systems for training astronauts for 
SpaceLab missions and engineers who test the Space Shuttle main 
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propulsion system. Although these tasks are quite different and 
are performed in very dissimilar environments, the same system 
architecture has proven to be adaptable to each. Below is a brief 
summary of the specific systems that have been built or are 
currently under development: 

A. PD/ICAT: [£ayload-assist module Deploys/ICAT System] 

A comprehensive intelligent computer-aided training system for use 
by Flight Dynamics Officers in learning to deploy PAM (Payload- 
Assist Module) satellites from the Space Shuttle. PD/ICAT 
contains four expert systems that cooperate via a blackboard 
architecture . 

B. WL/ICAT: [Vacuum Yent Line/ICAT System] 

A PC-based intelligent computer-aided training system for use by 
mission and payload specialists in learning to perform fault 
detection, isolation, and reconfiguration on the Spacelab WL 
system. WL/ICAT consists of an integrated expert system and 
graphical user interface. 

C. MPP/ICAT: [Main Propulsion Pneumatics/ICAT System] 

A comprehensive intelligent computer-aided training system for use 
by test engineers at NASA/Kennedy Space Center in learning to 
perform testing of the Space Shuttle Main Propulsion Pneumatics 
system. MPP/ICAT is currently under development and makes use of 
the same architecture as PD/ICAT. 

D. IPS/ICAT: [Instrument Pointing System/ICAT System] 

A comprehensive intelligent computer-aided training system for use 
by payload and mission specialists at NASA/Johnson Space Center 
and Marshall Space Flight Center in learning to utilize the IPS on 
Spacelab missions. IPS/ICAT is currently under development and 
makes use of the same architecture as PD/ICAT. 

III. A GENERAL ARCHITECTURE FOR INTELLIGENT TRAINING SYSTEMS 

The projects described in the previous section have served as 
vehicles to aid in the design and refinement of an architecture 
for intelligent training systems that has significant domain- 
independent elements and is generally applicable to training in 
procedural tasks common to the NASA environment. The ICAT system 
architecture is modular and consists of five basic components: 

• A user interface that permits the trainee to access the 
same information available to him in the the task 
environment and serves as a means for the trainee to take 
actions and communicate with the intelligent training 
system. 
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• A domain expert which can carry out the task using the same 
information that is available to the trainee and which also 
contains a list of "mal-rules" (explicitly identified 
errors that novice trainees commonly make) . 

• A training session manager which examines the actions taken 
by the domain expert (of both correct and incorrect actions 
in a particular context) and by the trainee and takes 
appropriate action(s). [Loftin, Baffes and Wang, 1988] 

• A trainee model which contains a history of the individual 
trainee's interactions with the system together with 
summary evaluative data. 

• A training scenario generator that designs increasingly- 
complex training exercises based on the knowledge of the 
domain expert, the current skill level contained in the 
trainee's model, and any weaknesses or deficiencies that 
the trainee has exhibited in previous interactions. 
[Loftin, Wang, and Baffes, 1988; Loftin, Wang, and Baffes, 
1989] 

Figure 1 contains a schematic diagram of the ICAT system. Note 
that provision is made for the user to interact with the system in 
two distinct ways and that a supervisor may also query the system 
for evaluative data on each trainee. The blackboard serves as a 
common repository of facts for all five system components. With 
the exception of the trainee model, each component makes 
assertions to the blackboard, and the expert system components 
look to the blackboard for facts against which their rules pattern 
match. A comprehensive effort has been made to clearly segregate 
domain-dependent from domain-independent components. 

IV. SYSTEM INTEGRATION 

The ICAT architecture described above was originally 
implemented in a Symbolics 3600 Lisp environment using Inference 
Corporation's ART for the rule-based components. The architecture 
is currently available for unix workstations. The user interface 
is implemented in X-Windows, the rule-based components in CLIPS 
[CLIPS is the acronym for a NASA-developed expert system shell 
written in C], and supporting code in C. 

V. TRAINING PERFORMANCE 

The original system developed with this architecture 
(PD / ICAT ) has been used by both expert and novice flight 
controllers at NASA/Johnson Space Center. An extensive 
investigation of the performance of novices using the system has 
been conducted. Figure 2 shows two measures of performance: (1) 

the time required to perform the nominal task as a function of the 
number of training experiences and (2) the number of errors made 
during the performance of the nominal task as a function of the 
number of training experiences. It is interesting to note that. 
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although the novices used in this investigation had very different 
levels of prior experience related to the task, all novices 
rapidly approached the same level of proficiency. 

VI. CONCLUSIONS 

A general architecture for ICAT systems has been developed 
and applied to the construction of three ICAT systems for very 
different tasks. Use by novices of an ICAT application built upon 
this architecture has shown impressive trainee performance 
improvements. With further refinement and extension, this 
architecture promises to provide a common foundation upon which to 
build intelligent training systems for many tasks of interest to 
the government, military, and industry. The availability of a 
robust architecture that contains many domain-independent 
components serves to greatly reduce the time and cost of 
developing new ICAT applications. 
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Figure 1 . A Schematic Diagram of the General Architecture 
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Performance of Novices Using the PD/ICAT System 


434 














